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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  as  part 
of  its  Training  for  Interactive  Distributed  Environments  project,  held  a  symposium  on  the  use  of 
training  games  in  the  U.S.  Military.  ARI  seeks  to  provide  the  Army  with  guidance  on  how 
games  can  be  used  for  military  training. 

On  5-6  November  2003,  ARI's  Advanced  Training  Methods  Research  Unit  held  a 
symposium  on  DoD’s  training  game  usage  through  a  contract  with  Booz  Allen  Hamilton. 
Presenters  from  all  Military  branches,  academia  and  the  private  sector  covered  topics  that 
included  effective  training  games,  course  integration,  barriers  to  implementation,  economical 
implementation,  and  evaluation.  Participants  stated  that  the  symposium  was  timely  and 
encouraged  periodic  follow  up  meetings. 

The  findings  were  shared  with  representatives  from  the  NATO  Research  and  Technology 
Organization  -  Modeling  and  Simulation  Group,  as  they  planned  for  an  October  2004  workshop 
on  the  use  of  commercial  games  for  military  purposes.  A  summary  of  this  symposium  was  also 
briefed  to  representatives  from  the  Research,  Development,  Experimentation 
Command/Simulation  and  Training  Technology  Center  (RDECOM/STTC),  the  Training  and 
Doctrine  Command  -  Training  Development  and  Analysis  Directorate  (TRADOC-TDAD),  the 
Army  Research  Laboratory  -  Human  Research  and  Engineering  Directorate  (ARL-HRED),  and 
the  Institute  for  Creative  Technologies  (ICT)  on  28  September  2004. 


A 


PAULA.  GADE 


Acting  Technical  Director 
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SYMPOSIUM  ON  PC-BASED  SIMULATIONS  AND  GAMING  FOR  MILITARY  TRAINING 


EXECUTIVE  SUMMARY 


Research  Requirement: 

There  is  an  increasing  use  of  PC-based  games  and  simulations  as  a  means  of  training 
within  the  U.S.  military,  and  many  of  these  efforts  are  being  done  in  isolation  with  little 
coordination  within  or  across  services.  To  bring  together  individuals  with  experience  in  the  use 
of  games  for  training  purposes  to  share  lessons  learned,  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  hosted  a  symposium  including  personnel  from  all  branches  of  the 
military  to  discuss  how  training  games  and  game-like  simulations  are  currently  being  used  by  the 
military. 

Method: 

The  symposium  consisted  of  1 1  presentations  given  by  training  representatives  from  all 
services,  military  training  researchers,  and  training  game  developers.  Additionally,  time  was 
allocated  for  group  discussion  among  the  50  symposium  participants  since  all  attendees  were 
experienced  in  the  use  of  training  games  for  military  purposes.  Note-takers  recorded  the 
presentations  and  the  discussions,  which  are  summarized  in  this  report.  At  the  beginning  of  the 
symposium,  five  organizing  questions  were  presented  to  the  group  to  focus  the  discussions.  The 
questions  were:  1)  how  are  training  games  used  effectively;  2)  what  interferes  with  training 
games;  3)  how  can  instructors/SMEs  facilitate  the  use  of  training  games;  4)  how  should  training 
game  developers  be  supported;  and  5)  what  are  barriers  to  training  games?  This  report  is 
organized  according  to  these  questions. 

Findings: 

The  primary  findings  were  based  on  the  five  organizing  questions  posed  at  the  beginning 
of  the  symposium.  The  highlights  of  the  discussion  were:  1)  training  games  need  to  be  used 
along  with  good  instructional  techniques;  2)  the  use  of  unrealistic  or  inappropriate  training 
scenarios  limit  the  effectiveness  of  training  games;  3)  trainers  are  required  to  guide  the  learner 
through  the  instruction,  and  the  games  are  training  tools  that  the  trainers  can  use;  4)  frequent  and 
reciprocal  communication  between  Instructors/SMEs  and  game  developers  is  critical  for 
successful  training  game  development;  and  5)  a  lack  of  policy  and  administrative  support  for 
training  games  is  a  barrier  to  the  implementation  of  training  games. 

Utilization  of  Findings: 

The  results  of  this  research  provide  information  beneficial  to  individuals  interested  in  the 
effective  use  of  training  games  for  military  purposes.  The  organizing  structure  of  the  symposium 
was  shared  with  representatives  from  the  NATO  Research  and  Technology  Organization  - 
Modeling  and  Simulation  Group,  as  they  planned  for  an  October  2004  workshop  on  exploiting 
commercial  games  for  military  use. 
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Introduction 


On  5-6  November  2003,  the  Advanced  Training  Methods  Research  Unit  (ATMRU)  of 
the  U.S.  Army  Research  Institute  for  Behavioral  and  Social  Sciences  hosted  a  symposium  on  the 
use  of  training  games  and  game-like  simulations  across  the  Department  of  Defense  (DoD).  The 
rationale  for  the  symposium  was  that  although  there  is  an  increasing  use  of  PC-based  games  and 
simulations  as  a  means  of  training  within  the  U.S.  Military,  many  of  these  efforts  are  being  done 
in  isolation  with  little  coordination  within  or  across  services.  The  objectives  of  this  symposium 
were  to  bring  personnel  from  all  branches  of  the  military  together  to  discuss  how  training  games 
and  game-like  simulations  are  currently  being  used  by  the  military,  to  identify  ways  they  may  be 
used  more  effectively,  and  to  help  develop  a  community  of  practitioners  to  facilitate  future 
development  and  effective  use  of  games  for  military  training.  Participating  in  this  two-day  event 
were  instructors  from  the  U.S.  Army,  Navy,  Air  Force,  and  Marines,  as  well  as,  training 
developers,  commercial  simulation/game  developers,  training  researchers  from  all  services,  and 
representatives  from  the  Office  of  the  Secretary  of  Defense. 

To  share  lessons  learned,  1 1  presentations  were  given  by  training  representatives  from  all 
services,  military  training  researchers,  and  training  game  developers.  A  list  of  the  presenters, 
their  affiliations,  and  the  title  of  presentations  are  displayed  in  Table  1.  In  addition,  time  was 
allocated  for  discussion  and  focus  groups  since  all  of  the  approximately  50  symposium 
participants  in  attendance  were  to  some  extent  involved  with  the  use  of  training  games  and 
desktop  simulations  (see  Appendix  A  for  a  full  list  of  all  participants).  This  discussion  time 
included  10-20  minute  question  periods  after  each  presentation,  a  90  minute  breakout  group 
sessions  at  the  end  of  the  first  day,  and  a  summary  review  of  the  symposium  proceedings  at  the 
end  of  the  second  day. 
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Table  1 

List  of  Presentations 


Name 


Presentation  title 


Mike  Crane 

U.S.  Air  Force  Academy 


‘The  Range  of  Training  Game  Options” 


Capt.  Erik  W .  Jilson  «The  uSMC's  PC-Based  Tactical  Decision  Simulations” 

TECOM,  U.S.  Marine  Corp 

CPT  Carl  Jacquet  “An  Instructor’s  View  on  the  Use  of  Training  Games  and 

U.S.  Military  Academy  Desktop  Simulations” 


Dr.  Scott  Beal 

U.S.  Army  Research 
Institute 
Rob  Sears 

Legless  Productions 
CMSgt  Teresa  Clark 

Nebraska  National  Guard 
CDR  Charles  Frye 
Naval  Air  Systems 
Command 
MAJ  Pat  Connors 

Battle  Command  Battle  Lab 
Fort  Leavenworth 
Jim  Dunnigan 

New  York,  NY  . 

Dr.  Ralph  Chatham 
Defense  Advanced 
Research  Projects  Agency 
Dr.  Harry  O’Neil 

University  of  Southern 
California  _ 


“Evaluation  of  a  Command  and  Control  Training  Game” 

“Development  of  Commercial  Platform  Training  Aids” 
“Economical  Interactions  in  Online  Training” 

“Training  Game  Research  and  Applications” 

"Providing  the  Warfighter  with  a  Low-Cost,  Low- 
Overhead,  PC-Based  Tactical  Trainer  -  A  Use  Case  " 

“Military  Training  Games” 

“Developing  a  Massive  Multiplayer  Online  Game  for 
Training” 

“Evaluating  the  Effectiveness  of  Training  Games” 


All  participants,  including  the  presenters,  were  asked  to  consider  a  set  of  questions  during 
the  symposium  in  an  effort  to  identify  the  “state  of  the  practice.”  The  five  organizing  questions 
are  presented  in  Table  2.  In  addition  to  gathering  responses  to  these  questions  at  the  symposium, 
the  questions  were  also  emailed  to  all  participants  one  week  after  the  symposium.  The 
participants  were  asked  to  respond  via  email  if  they  wanted  to  contribute  additional  comments 
that  were  not  covered  during  the  symposium.  In  an  effort  to  document  some  of  the  thoughts  and 
opinions  of  knowledgeable  persons  in  the  field  of  the  use  of  training  games  for  military  purposes, 
the  following  provides  a  summary  of  the  information  provided  both  during  the  symposium  and 
from  the  follow-up  email  responses,  organized  according  these  principle  questions. 
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Table  2 

Organizing  Questions  for  the  Symposium  _ ___ _ 

Question s  presented  before  discussion  groups _ _ _ 

1.  How  are  training  games  used  effectively? 

2.  What  interferes  with  training  games? 

3.  How  can  instructors/SMEs  facilitate  the  use  of  training  games? 

4.  How  should  training  game  developers  be  supported? 

_ 5.  What  are  barriers  to  training  games? _ 

The  information  presented  in  the  main  body  of  this  manuscript  covers  the  thoughts  and 
opinions  of  the  symposium  participants.  For  practical  purposes,  a  limit  of  50  participants  was  set 
for  the  symposium.  The  authors  admit  that  this  may  have  excluded  some  valuable  input. 
Therefore,  additional  information  on  the  topics  discussed  in  this  paper  is  presented  in  a  list  of 
resources  appearing  in  Appendix  B. 


Question  -  Topics 

How  are  Training  Games  Used  Effectively? 

This  question  was  interpreted  two  different  ways:  a)  what  makes  the  use  of  training 
games  an  effective  training  methodology,  and  b)  what  are  types  of  training  where  games  can  be 
used  effectively.  Both  interpretations  of  the  question  are  discussed  below. 

In  response  to  what  makes  training  games  an  effective  training  methodology,  the 
overwhelming  consensus  was  that  training  games  need  to  be  integrated  with  good  instructional 
techniques.  Training  games  are  instructional  tools.  As  with  all  types  of  tools  they  can  either  be 
used  properly  or  improperly,  which  tends  to  lead  to  positive  or  negative  outcomes,  respectively. 
Several  of  the  presenters  stated  that  one  of  the  reasons  training  games  can  be  effective  is  that 
they  promote  active  learning.  Training  games  allow  learners  to  actively  engage  in  the  content 
they  are  learning,  which  is  likely  to  produce  positive  learning  outcomes. 

The  instructional  methodology  that  was  mentioned  most  frequently  was  the  proper  use  of 
Instructor/SME  feedback  to  the  trainee.  For  training  games  this  feedback  can  be  presented  in 
various  forms,  ranging  from  global  to  specific.  Global  feedback  can  be  given  in  the  form  of  an 
after-action  review  (AAR)  after  the  game  has  been  completed,  or  more  immediate  specific 
feedback  can  also  be  provided  during  the  flow  of  the  game.  Both  global  and  specific  feedback 
can  promote  learning.  The  importance  of  the  instructor  providing  feedback  to  the  trainees  was  to 
highlight  important  lessons  learned  by  trainees  and  to  diminish  the  possibility  of  learning 
incorrect  procedures  from  the  game.  Without  instructors  monitoring  game  use,  many  participants 
stated  that  there  was  the  possibility  for  negative  training  to  occur. 

Other  beneficial  instructional  techniques  mentioned  were:  the  presence  of  progressive 
goals,  reliable  outcome  measures,  the  need  for  realistic  interfaces,  and  scenario  flexibility.  The 
instructional  techniques  for  games  have  not  yet  been  fully  explored  through  empirical  research, 
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but  the  research  is  beginning.  Symposium  participants  stated  that  basic,  effective  instructional 
methodology  should  apply  to  games,  although  more  research  is  needed  to  verify  specific 
instruction  methodologies  and  their  effectiveness  in  particular  training  game  contexts. 

In  response  to  the  question  of  the  types  of  training  for  which  games  can  be  used 
effectively,  the  majority  of  the  participants  stated  that  training  games  can  be  used  to  train 
cognitive  skills.  Most  of  the  actions  portrayed  in  games  are  not  conducted  with  a  game  interface 
that  is  identical  to  the  interface  used  in  a  real  world  context;  therefore,  the  physical  requirements 
for  the  game  may  not  transfer  as  well  to  the  real  world.  However,  the  cognitive  aspects  of  the 
game-tasks  may  transfer  better.  For  example,  if  the  training  objective  for  a  game  is  to  clear  a 
building,  the  trainees  may  learn  the  sequence  of  steps  (e.g.,  progressing  through  the  building  in  a 
specific  order)  required  to  clear  the  building,  but  they  probably  won’t  learn  the  actual  physical 
tasks  (e.g.,  running  through  halls  and  knocking  down  doors). 

Many  participants  stated  that  games  could  not  replace  live  fire  exercises  and  live 
simulations  such  as  those  conducted  at  the  National  Training  Center,  the  Joint  Readiness 
Training  Center,  and  other  facilities.  However,  many  participants  expressed  the  opinion  that 
training  games  could  be  useful  in  preparing  personnel  to  learn  some  of  the  prerequisite 
cognitively-based  skills  that  could  then  be  transferred  to  a  training  center.  Training  games  might 
bridge  the  gap  between  classroom  learning  and  live  training  exercises  thereby  enabling  more 
effective  use  of  the  latter  costly  resource. 

The  participants  also  stated  that  training  games  can  be  used  to  demonstrate  or  introduce 
general  concepts.  Some  of  the  presenters  who  were  instructors  stated  that  when  they  use  games 
to  demonstrate  concepts,  they  first  provide  some  background  information  on  a  particular  topic 
before  having  the  learners  use  the  game  to  “try-out”  the  concept.  This  allows  the  learners  to 
actively  engage  the  material.  This  approach  was  seen  as  a  way  to  provide  clear  concrete 
examples  in  a  format  that  is  far  more  compelling,  and  it  also  provides  a  different  means  for 
conveying  the  information  that  involves  the  learners  more  then  just  straight  lecture. 

Motivation  was  also  discussed  as  an  aspect  of  games  that  provided  instructional  value. 
Motivated  learners  are  more  likely  to  engage  in  the  training  task  than  unmotivated  learners,  and 
many  of  today’s  military  personnel  are  motivated  by  games.  In  order  to  benefit  from  increased 
trainee  motivation  based  on  the  use  of  training  games,  the  training  games  need  be  interesting  and 
engaging,  which  subsequently  promotes  continued  use  that  should  translate  to  increased  practice 
time  and  higher  levels  of  learning. 

Training  games  allow  instructors  to  overcome  some  of  the  constraints  of  time  and 
distance,  adding  to  the  utility  of  the  training  they  provide.  For  example,  to  take  a  class  to  the 
field  for  training  requires  travel  time  and  expense.  A  game-based  simulation  can  be  used  to 
augment  this  form  of  training,  providing  the  instructor  with  a  tool  to  run  quick  scenarios  where 
trainees  can  take  part  in  the  simulation  without  ever  leaving  the  classroom,  further,  a  training 
game  can  be  used  individually  by  students  at  their  own  convenience. 

There  are  other  benefits  to  controlling  time  and  distance.  With  regards  to  time,  the 
instructors  can  easily  “pause”  a  game  to  provide  feedback  or  additional  information  to  the 
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trainees.  The  instructor  also  has  the  ability  to  jump  around  the  virtual  environment  much  more 
quickly  than  a  real  environment,  allowing  them  to  monitor  trainees  from  multiple  perspectives. 
Another  benefit  of  controlling  distance  is  that  the  instructor  may  be  able  to  modify  the  scale, 
angle,  or  visual  perspective  of  what  is  observed,  which  allows  learners  to  get  a  better 
understanding  of  what  occurs  during  the  game  and  what  could  occur  in  the  real  world. 

What  Interferes  with  Training  Games? 

Interference  with  the  training  aspect  of  training  games  can  occur  in  a  variety  of  ways.  The 
interference  can  be  thought  of  as  a  matter  of  degree;  what  causes  the  game  to  fall  short  of  its 
training  potential  (limited  effectiveness)  or  what  keeps  training  games  from  being  used  at  all. 

The  most  frequently  mentioned  factor  that  may  limit  the  effectiveness  of  training  games 
was  the  use  of  unrealistic  or  inappropriate  training  scenarios.  The  scenario  must  match  the 
learning  objectives  for  the  training.  The  inability  to  edit  a  scenario  was  considered  a  major 
limitation  of  some  games  because  it  did  not  allow  the  instructor  to  customize  the 
game/simulation  for  specific  training  needs.  Instructors  stated  that  having  the  ability  to  modify 
scenarios  allows  them  to  create  targeted  training  that  can  focus  on  the  desired  learning 
objectives. 

It  was  also  noted  by  participants  that  the  possibility  of  negative  training  or  the  training  of 
improper  “stuff’  may  occur  with  some  training  games.  The  lack  of  SME  involvement  in  game 
development,  the  lack  of  instructor  involvement  (e.g.,  monitoring  trainee  actions  and  providing 
useful  feedback  to  trainees),  and  a  lack  of  clear  training  objectives  were  all  reasons  stated  for 
how  negative  training  could  occur  with  the  use  of  training  games. 

The  inclusion  of  extraneous  features  (bells  and  whistles  that  are  not  realistic  to  the 
training  scenario)  was  also  cited  as  interfering  with  training  effectiveness.  For  example,  some 
games  provide  a  hyper-reality,  in  that  they  make  information  or  resources  available  to  trainees 
that  an  ordinary  person  in  a  similar  real-life  situation  would  not  have  available  (e.g.,  god’s-eye 
views,  unlimited  ammunition,  and  coming  back  to  life  quickly  after  being  killed).  A  few  of  the 
participants  felt  that  the  inclusion  of  these  extraneous  features  might  lead  to  trainees  becoming 
reliant  on  the  additional  resources  that  won’t  always  be  available  to  them.  This  could  lead  to 
inappropriate  actions  when  trying  to  apply  what  they  learned  to  real-world  situations. 

Another  identified  interference  was  a  lack  of  “buy  in”  by  supervisors  and  administrators. 
This  kept  training  games  from  being  used  in  situations  where  they  might  have  been  effective.  As 
with  many  innovations,  there  is  initial  resistance,  but  with  demonstrated  usefulness  this 
resistance  is  minimized  over  time.  A  related  point  was  that  some  people  cringe  when  they  hear 
the  word  “games”  in  respect  to  military  training,  with  the  thought  that  games  are  fun  while 
training  is  serious.  For  this  reason,  some  of  the  participants  suggest  using  alternative  terms  like: 
training  simulations,  desktop  simulations,  or  PC-based  training  tools  to  minimize  this  difficulty. 

Another  barrier  to  the  effective  use  of  training  games  was  that  not  all  people  like  to  play 
games.  There  is  a  misconception  that  almost  everyone  in  the  current  generation  entering  military 
service  is  an  expert  with  video  games.  While  video  games  are  popular,  there  are  also  a  large 
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number  of  people  who  rarely  play  these  games.  The  issue  of  the  divide  between  those  with 
extensive  gaming  experience  and  those  who  are  relatively  new  to  such  games  was  discussed  and 
will  be  further  explored  in  the  “Questions  still  unanswered”  section. 

How  can  Instructors/SMEs  Facilitate  Use  of  Training  Games? 

Across  the  board,  participants  stated  that  instructors/SMEs  were  critical  to  the  success  of 
training  games.  While  some  game  developers  may  tout  their  game  as  a  stand-alone  training  tool, 
the  majority  of  the  attendees  stated  that  a  game  by  itself  would  not  necessarily  train.  Instructors 
are  needed  to  facilitate/guide  the  instruction.  This  section  covers  what  actions  instructors  can 
perform  in  order  to  maximize  the  positive  impact  of  the  training  experience. 

In  order  for  the  instruction  to  be  effective,  the  instructor  should  possess  a  high  level  of 
understanding  of  the  subject  matter.  In  addition,  the  instructor  must  be  familiar  with  the  specific 
game  being  used.  For  example,  how  to  set  up  and  run  the  game,  modify  scenarios,  and  address 
any  issues  as  needed  when  events  do  not  work  out  as  planned. 

When  using  a  training  game,  the  instructor  has  to  channel  students’  efforts  towards  the 
game  and  the  training  objectives.  The  instructor  should  ensure  that  the  training  objectives  are 
being  met  by  the  students.  The  goal  is  not  for  the  students  to  play  the  game  but  to  achieve  the 
training  objectives.  It  is  a  waste  of  time  if  a  student  excels  at  a  game  but  can  not  use  what  is 
learned  in  an  applied  setting.  Some  students  may  attempt  to  ‘  game  the  game,  which  is  finding 
an  alternative  way  to  score  well  or  reach  objectives  through  inappropriate  means.  Instructors 
need  to  make  sure  that  students  are  using  the  game  appropriately  and  learning  the  training 
objectives,  not  bypassing  them  and  therefore  bypassing  the  training. 

In  addition,  the  ability  to  provide  feedback  to  students  about  their  performance  was 
considered  important.  This  feedback  should  be  scaled  to  the  learner  and  the  task  at  hand.  For 
example,  when  a  learner  is  performing  a  novel  task,  they  may  need  more  directed  and  frequent 
feedback.  As  the  learner  becomes  more  skilled  at  particular  tasks,  the  feedback  can  be  scaled 
back. 

How  should  Training  Game  Developers  be  Supported? 

For  the  development  of  effective  training  games,  instructors/SMEs  need  to  provide 
support  to  the  game  developers.  Communication  between  instructors/SMEs  and  game  developers 
was  stated  as  critical  for  successful  training  game  development.  This  communication  should 
include  the  clear  identification  of  the  training  objectives  and  explanations  of  how  the  game  will 
be  used  to  train  students  (e.g.,  stand  alone  training  vs.  use  as  a  training  tool).  The  pedagogical 
focus  of  the  game  must  also  be  clearly  described;  otherwise  the  game  may  not  train  what  it 
should  train.  This  communication  needs  to  be  reciprocal,  in  that  game  developers  also  need  to 
clearly  communicate  their  capabilities  to  provide  the  training  game  functionality  expected  by  the 
instructors/SMEs. 

To  facilitate  this  mutual  communication,  one  suggestion  made  by  a  presenter  was  the 
development  of  a  prototypical  board  game  as  part  of  the  process  to  develop  a  PC-based  training 
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game.  The  development  of  a  board  game  allows  for  the  interested  parties  to  discuss  and  observe 
the  dynamics  of  the  game  before  a  lot  of  time  and  money  is  spent  writing  the  code  to  develop  the 
PC-based  version.  This  should  allow  the  developers  and  the  instructors/SMEs  to  clearly 
communicate,  because  they  would  be  able  to  demonstrate  what  they  are  thinking  with  the  board 
game.  It  also  allows  them  to  explore  different  ways  in  which  the  game  can  be  used. 

After  a  game  has  been  developed  (or  during  beta-testing)  communication  continues  to  be 
important  to  the  continued  improvement  of  the  game.  PC-based  games  can  be  modified  and  new 
scenarios  can  be  added,  increasing  the  quantity  of  training  content.  In  the  commercial  PC- 
gaming  world,  users  regularly  develop  "mods”  (modifications  to  the  game  scenarios).  This 
ability  to  develop  mods  is  a  strength  of  games  and  the  gaming  community  in  that  the  new  and 
varied  training  scenarios  can  be  developed  with  little  cost.  With  varied  training  scenarios, 
learners  may  better  generalize  and  transfer  the  skills  they  learn  from  one  setting  to  another. 

What  are  Barriers  to  Training  Games? 

The  noted  barriers  to  the  effective  use  of  training  games  fell  within  two  categories.  The 
first  set  dealt  with  how  the  game  was  implemented  in  the  instruction  or  factors  specific  to 
particular  games.  The  second  dealt  with  more  administrative  concerns,  things  external  to  the 
game/instructor  setting. 

Participants  cited  vague  training  objectives  as  a  barrier  that  limits  the  instructional 
effectiveness  of  training  games.  Unless  the  training  objectives  were  clear  and  the  game  was  well 
suited  for  these  particular  training  objectives,  appropriate  learning  was  unlikely  to  take  place. 
Another  barrier  to  the  effective  use  of  training  games  was  follow-through  by  instructors.  In  some 
instances  an  instructor  might  provide  a  game  and  then  just  let  students  play,  and  the  instructor 
might  not  monitor  the  training.  This  lack  of  instructor  monitoring  could  lead  to  negative  training, 
where  the  students  learn  inappropriate  procedures  from  the  game. 

Finally,  a  concern  over  the  time  spent  learning  how  to  play  the  game  was  mentioned.  For 
example,  if  it  takes  an  hour  to  learn  all  of  the  controls  of  the  game  and  then  another  hour  of 
playing  in  order  to  learn  material  that  can  be  equally  well  learned  through  10  minutes  of  lecture, 
then  the  game  would  not  be  a  time  efficient  means  for  training  this  particular  material.  However, 
if  the  two  hours  of  training  game  use  lead  to  increased  retention  and  ability  to  apply  the  training 
content  to  real-world  situations,  then  the  increased  time  may  be  justified. 

In  the  category  of  administrative  barriers,  the  lack  of  a  clear  policy  governing  the  use  of 
training  games  was  mentioned  as  a  barrier  to  their  implementation.  Several  of  the  participants 
mentioned  training  administrators  that  were  reluctant  to  embrace  this  new  form  of  training. 
Another  issue  mentioned  was  the  fear  that  the  use  of  training  games  will  lead  to  a  reduced 
training  budget.  The  need  for  better  technical  support  to  ensure  that  the  use  of  the  training  games 
run  without  technical  glitches  was  also  a  factor  identified  as  a  barrier  to  implementation. 
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Questions  Still  Unanswered 


There  were  a  wide  range  of  topics  discussed  regarding  the  questions  that  were  still 
unanswered  in  the  use  of  games  as  a  training  medium  for  military  purposes.  The  most  often 
mentioned  was  the  question  of  funding.  Where  will  the  funding  to  develop  needed  training 
games  come  from?  Most  of  the  games  currently  developed  by  DoD  are  more  experimental 
(proof-of-concepts)  to  show  that  a  specific  type  of  game  can  be  developed,  but  these  training 
games  do  not  regularly  make  it  to  the  classroom.  The  funding  for  implementation  of  the  game 
(e.g.,  producing  supporting  training  material  and  maintaining  or  modifying  the  game)  is  not 
allocated  as  part  of  the  research  and  development  process.  Right  now,  at  the  point  of  instruction 
many  of  the  games  used  are  commercial-off-the-shelf  (COTS)  that  meet  only  some  of  the 
training  requirements.  The  budget  for  these  games  involves  the  purchase  of  the  COTS  version, 
but  there  is  usually  little  to  no  money  available  for  game  development  or  for  modifying  games  so 
that  they  can  be  adapted  to  the  specific  training  objectives  needed  for  instruction. 

The  issue  of  the  lack  of  empirical  research  regarding  the  effectiveness  of  training  games 
was  also  raised.  Are  these  games  truly  effective  training  tools,  and  what  factors  can  maximize 
their  effectiveness?  While  there  seemed  to  be  near  consensus  that  games  can  be  effective  training 
tools,  very  little  empirical  research  has  confirmed  such  results.  Most  of  the  evidence  that  training 
games  can  be  effective  is  anecdotal  or  lacks  a  control  group  for  proper  comparison. 

The  place  that  training  games  will  take  in  the  overall  realm  of  training  options  also  needs 
clarification.  What  are  the  situational  variables  that  indicate  when  a  training  game  should  be  used 
to  augment  existing  training?  There  are  various  types  of  skills  (e.g.,  cognitive  and  motor)  and 
topic  areas  (e.g.,  tactics,  vehicle  maintenance,  and  medical  procedures)  that  may  or  may  not  be 
trained  effectively  by  particular  types  of  games.  For  example,  some  types  of  games  such  as  first- 
person-perspective  games  may  be  effective  for  training  cognitive  strategies  during  military 
exercises  but  may  not  be  effective  for  training  verbal  information.  This  type  of  research, 
identifying  the  relative  training  potential  of  different  types  of  games  for  training  different  types 
of  skills  or  topics,  has  yet  to  be  done. 

Not  everyone  has  extensive  experience  with  games,  and  that  could  influence  the 
effectiveness  of  games  as  a  training  medium  across  individuals.  The  initiation  of  non-gamers  into 
a  gaming  environment  was  a  concern  mentioned  during  the  symposium.  What  methods  should  be 
used  to  ensure  that  trainees  with  minimal  computer  gaming  experience  can  benefit  from  the 
training  as  well  as  experienced  gamers?  For  learners  with  minimal  gaming  experience,  there 
may  be  some  prerequisite  training  that  could  provide  the  necessary  foundation  so  that  the  learner 
could  take  full  advantage  of  game-based  learning. 

The  creation  of  clear  metrics  for  the  measurement  of  performance  in  the  training  game 
and  performance  in  the  process  of  the  game  was  also  raised  as  an  issue.  How  do  we  measure 
game  performance  to  determine  how  well  a  person  has  been  trained?  Some  games  have  a 
scoring  mechanism,  but  the  value  of  the  score  may  not  match  the  training  objectives  of  the 
instruction.  The  score  might  be  based  on  the  number  of  enemies  killed,  but  may  not  take  into 
consideration  the  number  of  friendlies  killed.  So  a  person  could  score  high  by  shooting 
everything  in  sight.  This  could  lead  to  learners  scoring  high  but  not  learning  what  they  are 


supposed  to  learn.  Also,  a  single  score  may  not  represent  what  a  trainee  learned  while  using  the 
game,  especially  if  the  score  is  for  a  team. 

While  games  have  been  used  for  training  purposes  for  decades  and  are  becoming  more 
popular  in  their  use,  we  are  still  in  the  infancy  stage  of  empirically  determining  the  factors  and 
training  techniques  with  which  we  can  maximize  the  use  of  games  for  effective  training. 
Additional  research  and  experience  must  be  accumulated  to  answer  some  of  the  questions  posed 
above.  Also,  while  answering  some  of  these  questions,  new  questions  will  assuredly  arise. 

Summary 

During  the  symposium,  the  questions  provided  an  organizing  structure  for  the 
presentations  and  discussion.  Within  this  structure,  several  dominant  themes  emerged  and  are 
highlighted  below.  The  overarching  theme  was  that  training  games  can  be  useful  for  military 
training  purposes,  but  there  are  still  considerable  obstacles  to  be  overcome. 

•  PC-based  simulations  and  games  need  to  be  treated  as  a  training  delivery  system  that 
includes  instructor/SME  support/monitoring,  supporting  training  material,  in  addition  to 
the  game  itself.  Today’s  training  games  can  not  provide  quality  training  in  isolation;  they 
must  be  part  of  a  sequence  of  instruction  that  may  include  the  learning  of  basic  concepts 
which  can  then  be  applied  in  a  gaming  environment. 

•  Sound  instructional  design  must  be  used  whereby  learning  objectives  are  provided  by 
military  SMEs  working  with  game  developers  to  ensure  these  training  objectives  are 
supported  by  the  game  scenarios.  Game  engine-based  simulations  should  also  provide 
feedback  to  the  trainee  and  incorporate  some  form  of  AAR.  In  addition,  instructors 
should  provide  additional  feedback  to  the  trainee  to  ensure  that  only  correct  military 
procedures  and  performances  are  reinforced.  Soldiers  should  not  learn  or  practice 
incorrect  behaviors,  which  could  have  negative  operational  consequences. 

•  Presenters  also  stressed  that  the  game-based  simulations  do  not  train  in  isolation;  trainers 
train,  and  games  are  just  a  training  tool  they  can  use.  At  this  point,  games  should  be  used 
as  training  tools  along  with  instructional  guidance  provided  by  trainers.  Trainers  should 
know  the  game,  and  know  how  the  game  supports  achievement  of  the  military  training 
objectives.  Trainers  can  then  meaningfully  interact  and  coach  the  students  to  enhance  the 
value  of  the  game-based  simulation. 

•  There  is  a  need  for  objective  evaluation  of  the  training  effectiveness  of  games.  There  is  a 
wealth  of  anecdotal  evidence  to  support  the  use  of  various  training  games,  but  a  dearth  of 
empirical  evidence.  Research  needs  to  identify  and  assess  where,  when,  and  how  training 
games  can  be  used  effectively. 

Training  games  have  a  wide  variety  of  potential  applications;  they  can  be  used  to  provide 
and  reinforce  initial  training;  and  they  can  provide  practice,  rehearsal,  and  refresher  training. 
Game-based  simulations  can  be  used  for  training  a  vast  spectrum  of  military  skills  and  tasks  on 
an  “any  time,  anywhere”  basis  from  flight  simulations  to  periscope  visual  training  to  platoon 
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leadership  to  intravenous  needle  insertions.  The  potential  for  training  games  is  high,  but 
considerable  work  remains  to  be  done  to  reach  that  potential. 
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Appendix  A 


Table  3.  List  of  presenters  and  attendees 


NAME  ORGANIZATION 


Abell,  Millie 

TRADOC 

Allen,  Robert 

NAVAIR 

Armstrong,  Wayman 

Engineering  &  Computer  Simulations 

Beal ,  Scott 

ARI,  EFRU 

Belanich,  Jim 

ARI,  ATMRU 

Blunt,  Rick 

Booz  Allen  Hamilton 

Boelter,  Ronald 

TDCDDL 

Cabrera,  Nancy 

MITRE 

Chatham,  Ralph 

DARPA 

Clark,  CMSgt  Teresa 

Nebraska  National  Guard 

Connors ,  MAJ  Pat 

BCBL,  Leavenworth 

Crane,  Michael 

Air  Force  Academy 

Dressel,  Doug 

ARI,  ATMRU 

Dunnigan,  Jim 

Freeman,  Mike 

OSD,  ADL  Initiative 

Frye,  CDR  Charles 

NAVAIR 

Garris,  Rosemary 

NAVAIR 

Geertsen,  Mike 

Microsoft 

Gehrki,  Frank 

Alion 

Hiller,  Jack 

Northrup  Grumman 

Howse,  Bill 

ARI,  RWARU 

Jacquet,  CPT  Carl 

USMA 

Jilson,  Capt.  Erik 

TECOM,  Quantico 

Jones,  Steve 

MITRE 

Juntif,  LTC  George 

Army  Game  Project 

Kenny,  Patrick 

Soar  Technology 

King,  Ben 

Konya,  Steve 

IDA 

Lawson,  John 

DRA 

Leser,  Jeff 

Command  &  General  Staff  College 

Lunsford,  Jim 

MAK  Technologies 

Mathes,  Capt.  Don 

USMC 

McCarley,  Britt 

TRADOC  Military  History 

(table  continues ) 
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Table  3.  ( continued ) 


NAME 

ORGANIZATION 

Morrison,  John 

IDA 

Moses,  Frank 

ARI,  ATMRU 

Mullin,  Laura 
Nichols,  Paul 

ARI,  ATMRU 

Oerlert,  Mark 

Booz  Allen  Hamilton 

O'Neil,  Harry 

use 

Perez,  Ray 

ONR 

Perla,  Peter 

Center  for  Naval  Analyses 

Peterson,  Charlie 

Microsoft 

Portis,  Frederick 

soc 

Quinkert,  Kathy 

ARI,  TRADOC 

Robar,  Jason 

Jason  Robar  Consulting 

Sams,  Michelle 

Teknowledge 

Sears,  Rob 

Legless  Productions 

Singer,  Mike 

ARI,  SSRU 

Smith,  Brent 

Tiller,  John 
Velazquez,  Luis 

Engineering  &  Computer  Simulations 

Wisher,  Robert 

OSD 
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Appendix  B  -  Resource  List 
Web-based  Resources 

Department  of  Defense  Game  Developer’s  Community  (www.dodgamecommunity.com).  This 

DoD  funded  website  is  designed  to  bring  together  the  community  of  game  developers  and 
game  users  within  the  U.  S.  military. 

Marine  Technology  and  Education  Command  site  that  covers  the  use  of  gaming  technology  for 
training  (http://www.tecom.usmc.mil/techdiv/rrK/ITKHome/ITKHome.htm) 

Milgames  listserv  -  to  join,  send  an  email  to  milgames-subscribe@yahoogroups.com 

Modeling  and  Simulation  Offices: 

•  Defense  Modeling  and  Simulation  Office  (https://www.dmso.mil) 

•  Army  Model  &  Simulation  Office  (AMSO)  (http://www.amso.army.mil/) 

•  Army  Simulation.  Training  &  Instrumentation  Command  (STRICOM) 
(www.peostri.army.mil) 

•  Naval  Air  Systems  Command  Training  Systems  Division  (NAVAIR-TSD) 
(http://www.ntsc.navy.mil/) 

•  Air  Force  Agency  for  Modeling  &  Simulation  (AFAMS)  (http://www.afams.af.mil) 

•  Defense  Advanced  Research  Project  Agency  -  Training  Superiority  (DARWARS) 
(http://www.darpa.mil/dso/thrust/biosci/training_super.htm) 

Serious  Games  Initiative  (www.seriousgames.org)  -  focuses  on  the  use  of  games  in  education, 
training,  health,  and  public  policy. 

Training  with  Games 

Gee,  James  Paul  (2003).  What  video  games  have  to  teach  us  about  learning  and  literacy. 
Palgrave  Macmillan:  New  York. 

Greenfield,  Patricia  M.  (1984).  Mind  and  Media:  The  Effects  of  Television,  Video  Games  and 
Computers.  Cambridge,  MA:  Harvard  University  Press. 

Prensky,  M.  (2001).  Digital  Game-Based  Learning.  New  York:  McGraw-Hill. 

Herz,  J.  C.,  (1997).  Joystick  nation:  how  videogames  ate  our  quarters,  won  our  hearts,  and 
rewired  our  minds.  New  York:  Little,  Brown  and  Company. 

Herz,  J.  C.  &  Macedonia,  M.  R.  (2002).  Computer  games  and  the  military:  Two  views.  Defense 
Horizons,  11.  Retrieved  at  http://www.ndu.edu/inss/DefHor/DHll/DHll.htm  on 
7/8/2003. 

Taylor,  J.  L.  &  Walford,  R.  (1978).  Learning  and  the  simulation  game.  Beverly  Hills,  CA:  Sage 
Publications. 
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War  Games 


Dunnigan,  J.  F.  (1992).  The  complete  wargames  handbook :  how  to  play,  design,  and  find  them. 
New  York:  Morrow. 

Featherstone,  D.  F.  (1962).  War  Games.  London:  Stanley  Paul  &  Co. 
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